{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. 什么是Alpha101?\n",
    "*用数学公式找到Alpha机会*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 目录\n",
    "\n",
    "1. Alpha101是什么？\n",
    "2. Alpha101用到的算法有哪些？\n",
    "3. Alpha001-010怎么写？\n",
    "4. 如何用TA_Lib设计新的Alpha因子？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alpha101是什么？\n",
    "WorldQuant根据数据挖掘的方法发掘了101个alpha，据说里面 80% 的因子仍然还行之有效并运行在他们的投资策略中。Alpha101给出的公式，也就是计算机代码101年真实的定量交易Alpha。他们的平均持有期大约范围0.6 - 6.4天。平均两两这些Alpha的相关性较低,为15.9%。回报是与波动强相关，但对换手率没有明显的依赖性，直接确认较早的间接经验分析结果。我们从经验上进一步发现换手率对alpha相关性的解释能力很差。\n",
    "\n",
    "PDF下载：\n",
    "\n",
    "Python代码下载："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alpha101主要元素有什么？\n",
    "### 1. 因子组成元素\n",
    "- 价量因子（52/101）：\n",
    "    - HLOC\n",
    "    - ADV\n",
    "    - VWAP\n",
    "    - Volume\n",
    "- 价格波动因子（21/101）:\n",
    "    - HLOC\n",
    "    - Return\n",
    "    - STD()\n",
    "- 组合因子(8/101): 价量与价波因子组合\n",
    "- 市值因子（1/101）:\n",
    "    - Return\n",
    "    - Cap\n",
    "- 板块组合因子（19/101）: 板块分类结合价量与价波因子\n",
    "\n",
    "### 2. 函数与运算符\n",
    "- 'x?y:z'是指x为True，返回y，否则返回z。\n",
    "- Rank是指横向的品种间排序。\n",
    "- ts_Rank是指纵向的时间序列排序。\n",
    "\n",
    "详细参考原文PDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alpha101用到的算法有哪些？\n",
    "1. 编制函数需要的算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 1. 编制函数需要的算法，\n",
    "#coding=utf-8\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.stats import rankdata\n",
    "\n",
    "# 计算alpha101时会使用的函数\n",
    "def ts_sum(df,window=10):\n",
    "    return df.rolling(window).sum()\n",
    "\n",
    "def ts_mean(df,window=10):\n",
    "    return df.rolling(window).mean()\n",
    "\n",
    "def stddev(df,window=10):\n",
    "    return df.rolling(window).std()\n",
    "\n",
    "def correlation(x,y,window=10):\n",
    "    return x.rolling(window).corr(y)\n",
    "\n",
    "def covariance(x,y,window=10):\n",
    "    return x.rolling(window).cov(y)\n",
    "\n",
    "def rolling_rank(na):\n",
    "    return rankdata(na)[-1]\n",
    "\n",
    "def ts_rank(df, window=10):\n",
    "    return df.rolling(window).apply(rolling_rank)\n",
    "\n",
    "def rolling_prod(na):\n",
    "    return na.prod(na)\n",
    "\n",
    "def product(df,window=10):\n",
    "    return df.rolling(window).apply(rolling_prod)\n",
    "\n",
    "def ts_min(df,window=10):\n",
    "    return df.rolling(window).min()\n",
    "\n",
    "def ts_max(df,window=10):\n",
    "    return df.rolling(window).max()\n",
    "\n",
    "def delta(df,period=1):\n",
    "    return df.diff(period)\n",
    "\n",
    "def delay(df,period=1):\n",
    "    return df.shift(period)\n",
    "\n",
    "def rank(df):\n",
    "    return df.rank(axis=1, pct=True)\n",
    "\n",
    "def scale(df,k=1):\n",
    "    return df.mul(k).div(np.abs(df).sum())\n",
    "\n",
    "def ts_argmax(df,window=10):\n",
    "    return df.rolling(window).apply(np.argmax)+1\n",
    "\n",
    "def ts_argmin(df,window=10):\n",
    "    return df.rolling(window).apply(np.argmin)+1\n",
    "\n",
    "def decay_linear(df,period=10):\n",
    "    if df.isnull().values.any():\n",
    "        df.fillna(method='ffill',inplace=True)\n",
    "        df.fillna(method='bfill',inplace=True)\n",
    "        df.fillna(value=0, inplace=True)\n",
    "    return pd.DataFrame(\n",
    "        {name: ta.WMA(item.values, period) for name, item in df.iteritems()},\n",
    "        index=df.index\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alpha001-010怎么写？\n",
    "2. 定义计算Alpha的类\n",
    "3. 编制因子的函数\n",
    "4. 传入股票池数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 2. 定义计算alpha值的类\n",
    "class alphas(object):\n",
    "    def __init__(self, pn_data):\n",
    "        \"\"\"\n",
    "        :传入参数 pn_data: pandas.Panel\n",
    "        \"\"\"\n",
    "        # 获取历史数据\n",
    "        self.open = pd.DataFrame(pn_data.minor_xs('open'), dtype=np.float64)\n",
    "        self.high = pd.DataFrame(pn_data.minor_xs('high'), dtype=np.float64)\n",
    "        self.low = pd.DataFrame(pn_data.minor_xs('low'), dtype=np.float64)\n",
    "        self.close = pd.DataFrame(pn_data.minor_xs('close'), dtype=np.float64)\n",
    "        self.volume = pd.DataFrame(pn_data.minor_xs('volume'), dtype=np.float64)\n",
    "        self.returns = pd.DataFrame(self.close.pct_change())\n",
    "        self.adv = ts_mean(self.volume, 10)\n",
    "        self.vwap = ts_sum(self.close*self.volume, 10)/ts_sum(self.volume, 10)\n",
    "\n",
    "# 3. 编制因子的函数\n",
    "    \n",
    "    #   alpha001:(rank(Ts_ArgMax(SignedPower(((returns < 0) ? stddev(returns, 20) : close), 2.), 5)) -0.5)\n",
    "    def alpha001(self):\n",
    "        inner = self.close\n",
    "        inner[self.returns < 0] = stddev(self.returns, 20)\n",
    "        alpha = rank(ts_argmax(inner ** 2, 5))\n",
    "        return alpha\n",
    "    \n",
    "    #  alpha002:(-1 * correlation(rank(delta(log(volume), 2)), rank(((close - open) / open)), 6))\n",
    "    def alpha002(self):\n",
    "        alpha = -1 * correlation(rank(delta(np.log(self.volume), 2)), rank((self.close - self.open) / self.open), 6)\n",
    "        return alpha.replace([-np.inf, np.inf], np.nan)\n",
    "\n",
    "    # alpha003:(-1 * correlation(rank(open), rank(volume), 10))\n",
    "    def alpha003(self):\n",
    "        alpha = -1 * correlation(rank(self.open), rank(self.volume), 10)\n",
    "        return alpha.replace([-np.inf, np.inf], np.nan)\n",
    "\n",
    "    # alpha004: (-1 * Ts_Rank(rank(low), 9))\n",
    "    def alpha004(self):\n",
    "        alpha = -1 * ts_rank(rank(self.low), 9)\n",
    "        return alpha\n",
    "    \n",
    "    # alpha005:(rank((open - (sum(vwap, 10) / 10))) * (-1 * abs(rank((close - vwap)))))\n",
    "    def alpha005(self):\n",
    "        alpha = (rank((self.open - (ts_sum(self.vwap, 10) / 10))) * (-1 * np.abs(rank((self.close - self.vwap)))))\n",
    "        return alpha\n",
    "    \n",
    "    # alpha006: (-1 * correlation(open, volume, 10))\n",
    "    def alpha006(self):\n",
    "        alpha = -1 * correlation(self.open, self.volume, 10)\n",
    "        return alpha\n",
    "        \n",
    "    # alpha007: ((adv20 < volume) ? ((-1 * ts_rank(abs(delta(close, 7)), 60)) * sign(delta(close, 7))) : (-1* 1))\n",
    "    def alpha007(self):\n",
    "        adv20 = ts_mean(self.volume, 20)\n",
    "        alpha = -1 * ts_rank(abs(delta(self.close, 7)), 60) * np.sign(delta(self.close, 7))\n",
    "        alpha[adv20 >= self.volume] = -1\n",
    "        return alpha\n",
    "\n",
    "    # alpha008: (-1 * rank(((sum(open, 5) * sum(returns, 5)) - delay((sum(open, 5) * sum(returns, 5)),10))))\n",
    "    def alpha008(self):\n",
    "        alpha = -1 * (rank(((ts_sum(self.open, 5) * ts_sum(self.returns, 5)) -\n",
    "                          delay((ts_sum(self.open, 5) * ts_sum(self.returns, 5)), 10))))\n",
    "        return alpha\n",
    "    \n",
    "    # alpha009:((0 < ts_min(delta(close, 1), 5)) ? delta(close, 1) : ((ts_max(delta(close, 1), 5) < 0) ?delta(close, 1) : (-1 * delta(close, 1))))\n",
    "    def alpha009(self):\n",
    "        delta_close = delta(self.close, 1)\n",
    "        cond_1 = ts_min(delta_close, 5) > 0\n",
    "        cond_2 = ts_max(delta_close, 5) < 0\n",
    "        alpha = -1 * delta_close\n",
    "        alpha[cond_1 | cond_2] = delta_close\n",
    "        return alpha\n",
    "\n",
    "    # alpha010: rank(((0 < ts_min(delta(close, 1), 4)) ? delta(close, 1) : ((ts_max(delta(close, 1), 4) < 0)? delta(close, 1) : (-1 * delta(close, 1)))))\n",
    "    def alpha010(self):\n",
    "        delta_close = delta(self.close, 1)\n",
    "        cond_1 = ts_min(delta_close, 4) > 0\n",
    "        cond_2 = ts_max(delta_close, 4) < 0\n",
    "        alpha = -1 * delta_close\n",
    "        alpha[cond_1 | cond_2] = delta_close\n",
    "        return alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "one:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    0.60    0.90    0.90    0.60    0.35    0.90    0.15    0.15   \n",
      "2016-12-29    0.45    0.75    0.75    0.45    0.15    0.95    0.45    0.95   \n",
      "2016-12-30    0.30    0.65    0.65    0.30    0.90    0.90    0.30    0.90   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    0.60    0.35  \n",
      "2016-12-29    0.45    0.15  \n",
      "2016-12-30    0.30    0.30  \n",
      "two:               000001    000002    000089    000402    000858    000895  \\\n",
      "date                                                                     \n",
      "2016-12-28 -0.441017 -0.782143 -0.173049 -0.764651  0.509797 -0.113961   \n",
      "2016-12-29 -0.340427 -0.834709  0.204831 -0.760374  0.514727 -0.360704   \n",
      "2016-12-30 -0.396615 -0.892770  0.384995 -0.627246  0.068359 -0.442627   \n",
      "\n",
      "              600006    600029    600036    600050  \n",
      "date                                                \n",
      "2016-12-28 -0.195710 -0.529655 -0.050030 -0.058135  \n",
      "2016-12-29 -0.165395 -0.322351 -0.032588 -0.365148  \n",
      "2016-12-30  0.146124 -0.502836  0.280158  0.154897  \n",
      "three:               000001  000002        000089  000402  000858  000895    600006  \\\n",
      "date                                                                           \n",
      "2016-12-28 -0.000005     NaN  8.829785e-08     NaN     NaN     NaN -0.229014   \n",
      "2016-12-29 -0.000005     NaN  2.648935e-07     NaN     NaN     NaN -0.386859   \n",
      "2016-12-30 -0.000005     NaN  1.851112e-07     NaN     NaN     NaN -0.175055   \n",
      "\n",
      "              600029    600036  600050  \n",
      "date                                    \n",
      "2016-12-28 -0.237023  0.000002     NaN  \n",
      "2016-12-29 -0.037113  0.000002     NaN  \n",
      "2016-12-30 -0.151620  0.000002     NaN  \n",
      "four:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    -5.0    -5.0    -5.0    -5.0    -5.0    -5.0    -3.0    -6.0   \n",
      "2016-12-29    -5.0    -5.0    -5.0    -5.0    -5.0    -5.0    -3.5    -6.0   \n",
      "2016-12-30    -5.0    -5.0    -5.0    -5.0    -5.0    -5.0    -3.5    -6.0   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    -5.0    -6.0  \n",
      "2016-12-29    -5.0    -5.5  \n",
      "2016-12-30    -5.0    -5.5  \n",
      "five:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28   -0.30   -0.18   -0.24   -0.56   -0.01   -0.08   -0.45   -0.56   \n",
      "2016-12-29   -0.48   -0.21   -0.15   -0.42   -0.01   -0.36   -0.32   -0.45   \n",
      "2016-12-30   -0.45   -0.04   -0.32   -0.36   -0.01   -0.70   -0.24   -0.63   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28   -0.09    -1.0  \n",
      "2016-12-29   -0.04    -1.0  \n",
      "2016-12-30   -0.15    -0.4  \n",
      "six:               000001    000002    000089    000402    000858    000895  \\\n",
      "date                                                                     \n",
      "2016-12-28 -0.867299  0.120440  0.221516 -0.010639 -0.409205  0.044966   \n",
      "2016-12-29 -0.660740  0.046528  0.190947 -0.009467 -0.423415  0.127706   \n",
      "2016-12-30 -0.788309 -0.185357  0.130481  0.003820 -0.312593  0.151085   \n",
      "\n",
      "              600006    600029    600036    600050  \n",
      "date                                                \n",
      "2016-12-28 -0.338414 -0.061250 -0.679120 -0.283615  \n",
      "2016-12-29 -0.208193 -0.075150 -0.286022 -0.138658  \n",
      "2016-12-30 -0.229996  0.035984 -0.293132  0.075360  \n",
      "seven:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0   \n",
      "2016-12-29    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0   \n",
      "2016-12-30    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0    -1.0   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    -1.0    -1.0  \n",
      "2016-12-29    -1.0    -1.0  \n",
      "2016-12-30    -1.0    -1.0  \n",
      "eight:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    -0.3    -1.0    -0.6    -0.9    -0.1    -0.8    -0.7    -0.5   \n",
      "2016-12-29    -0.3    -1.0    -0.6    -0.8    -0.1    -0.9    -0.4    -0.7   \n",
      "2016-12-30    -0.5    -1.0    -0.4    -0.6    -0.8    -0.9    -0.2    -0.3   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    -0.2    -0.4  \n",
      "2016-12-29    -0.5    -0.2  \n",
      "2016-12-30    -0.7    -0.1  \n",
      "nine:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    0.02    0.22    0.14    0.15  -0.421   0.170    0.11   -0.06   \n",
      "2016-12-29   -0.02    0.36    0.07    0.09  -0.059  -0.274    0.01   -0.02   \n",
      "2016-12-30   -0.02    0.29   -0.10    0.04  -0.617  -0.237    0.04   -0.03   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    0.09   -0.14  \n",
      "2016-12-29    0.13    0.15  \n",
      "2016-12-30   -0.11    0.23  \n",
      "ten:             000001  000002  000089  000402  000858  000895  600006  600029  \\\n",
      "date                                                                         \n",
      "2016-12-28    0.02    0.22    0.14    0.15  -0.421   0.170    0.11   -0.06   \n",
      "2016-12-29   -0.02    0.36    0.07    0.09  -0.059  -0.274    0.01   -0.02   \n",
      "2016-12-30   -0.02    0.29   -0.10   -0.04  -0.617  -0.237   -0.04   -0.03   \n",
      "\n",
      "            600036  600050  \n",
      "date                        \n",
      "2016-12-28    0.09   -0.14  \n",
      "2016-12-29    0.13    0.15  \n",
      "2016-12-30   -0.11    0.23  \n"
     ]
    }
   ],
   "source": [
    "# 4. 传入股票池数据\n",
    "if __name__ == '__main__':\n",
    "    import pandas as pd\n",
    "    import tushare as ts\n",
    "\n",
    "    codes = ['000001', '601318', '600029', '000089', '000402', \n",
    "             '000895', '600006', '000858', '600036', '600050']\n",
    "    stocks_dict = {}\n",
    "    for c in codes:\n",
    "        stock = ts.get_k_data(c, start='2016-01-01', end='2016-12-31', ktype='D', autype='qfq')\n",
    "        stock.index = pd.to_datetime(stock['date'], format='%Y-%m-%d')\n",
    "        stock.pop('date')\n",
    "        stocks_dict[c] = stock\n",
    "    \n",
    "    pn = pd.Panel(stocks_dict)\n",
    "\n",
    "# 计算cmt001-010的值\n",
    "print 'one:', alphas(pn).alpha001().tail(3)\n",
    "print 'two:', alphas(pn).alpha002().tail(3)\n",
    "print 'three:', alphas(pn).alpha003().tail(3)\n",
    "print 'four:', alphas(pn).alpha004().tail(3)\n",
    "print 'five:', alphas(pn).alpha005().tail(3)\n",
    "print 'six:', alphas(pn).alpha006().tail(3)\n",
    "print 'seven:', alphas(pn).alpha007().tail(3)\n",
    "print 'eight:', alphas(pn).alpha008().tail(3)\n",
    "print 'nine:', alphas(pn).alpha009().tail(3)\n",
    "print 'ten:', alphas(pn).alpha010().tail(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 如何用TA_Lib设计新的Alpha因子？\n",
    "加入slope技术指标"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              000001    000002    000089    000402    000858    000895  \\\n",
      "date                                                                     \n",
      "2016-12-26  0.035455  0.308667  0.027515  0.006545  0.144055  0.023727   \n",
      "2016-12-27  0.028909  0.192182  0.013636 -0.022121  0.141430  0.020703   \n",
      "2016-12-28  0.020667  0.119152  0.014970 -0.027273  0.190570  0.040285   \n",
      "2016-12-29  0.017212  0.049879  0.021394 -0.022303  0.232873  0.048182   \n",
      "2016-12-30  0.010606 -0.030303  0.015636 -0.018061  0.216691  0.035261   \n",
      "\n",
      "              600006    600029    600036    600050  \n",
      "date                                                \n",
      "2016-12-26 -0.055697 -0.035818  0.137212 -0.112848  \n",
      "2016-12-27 -0.048061 -0.037879  0.104061 -0.131030  \n",
      "2016-12-28 -0.022848 -0.029697  0.065636 -0.127636  \n",
      "2016-12-29  0.005394 -0.016606  0.058424 -0.098545  \n",
      "2016-12-30  0.021394 -0.004727  0.040121 -0.050545  \n"
     ]
    }
   ],
   "source": [
    "import talib as ta\n",
    "import numpy as np\n",
    "\n",
    "def slope(df, period=10):\n",
    "    return pd.DataFrame(\n",
    "        {name: ta.LINEARREG_SLOPE(item.values, period) for name, item in df.iteritems()},\n",
    "        index=df.index\n",
    "        )\n",
    "\n",
    "class alphas(object):\n",
    "    def __init__(self, pn_data):\n",
    "        if pn_data.isnull().values.any():\n",
    "            pn_data.fillna(method='ffill',inplace=True)\n",
    "        self.close = pd.DataFrame(pn_data.minor_xs('close'), \n",
    "                                  dtype=np.float64)\n",
    "\n",
    "# 自制因子\n",
    "    def slope001(self):\n",
    "        alpha = -1 * slope(self.close)\n",
    "        return alpha\n",
    "\n",
    "print alphas(pn).slope001().tail(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 作业\n",
    "下载Alpha101完整代码研究，并设计有效的Alpha因子，导入Alphalens计算绩效。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
